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Abstract. Supplement 1 to GUM (GUM-SI) recommends the use of maximum 

entropy principle (MaxEnt) in determining the probability distribution of a 

quantity having specified properties, e.g., specified central moments. When we 

C^ , only know the mean value and the variance of a variable, GUM-SI prescribes a 

O ' Gaussian probability distribution for that variable. When further information is 

' !^ ' available, in the form of a finite interval in which the variable is known to lie. 



we indicate how the distribution for the variable in this case can be obtained. A 
Gaussian distribution should only be used in this case when the standard deviation 
is small compared to the range of variation (the length of the interval). In general, 
when the interval is finite, the parameters of the distribution should be evaluated 
numerically, as suggested by I. Lira [Metrologia, 2009, 46, L27]. Here we note that 
the knowledge of the range of variation is equivalent to a bias of the distribution 
►^ ' toward a flat distribution in that range, and the principle of minimum KuUback 

_^ , entropy (mKE) should be used in the derivation of the probability distribution 

rather than the MaxEnt, thus leading to an exponential distribution with non 
Gaussian features. Furthermore, up to evaluating the distribution negentropy, 
we quantify the deviation of mKE distributions from MaxEnt ones and, thus, 
we rigorously justify the use of GUM-SI recommendation also if we have further 
f-f^ ^ information on the range of variation of a quantity, namely, provided that its 

f**^ , standard uncertainty is sufficiently small compared to the range. 
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Supplement 1 to GUM (GUM-SI) [Ij provides assignments of probability density 
functions for some common circumstances. In particular, it is stated that if we know 
only the mean value x and the variance aj^ of a certain quantity X, we should assign 
a Gaussian probability distribution to that quantity, according to the principle of 
maximum entropy (MaxEnt) [2l[3]. The derivation is quite simple, as one has to look 
for the distribution p{x) maximizing the Shannon entropy: 



S[p] = dxp{x) \ogp{x) , (1) 



E, 



which is given by: 



p{x) — exp{— Ao — \ix — X2X^} , (2) 

where the values of the coefficients A^. should be determined to satisfy the constraints: 

dx p{x) x'' ^ Mk , (3) 

R 

with: 

Mo = l, Mi^x, M2^<Jx+x^- (4) 

However, sometimes we also know the range of the possible values of the quantity 
X. Two relevant examples are given by the phase-shift in interferometry, which 
is topologically confined in a 27r-window, and by the displacement amplitude of a 
harmonic oscillator, whose range of variation is dictated by energy constraints. In this 
case, it has been noticed by I. Lira in [4, that a Gaussian probability distribution with 
support on the real axis can be rigorously justified only if the standard uncertainty 
is sufficiently small with respect to the range of variation of the quantity. More in 
details, if we have any information about the range of variation, then this information 
should be employed in deriving the distribution maximizing the entropy as well as in 
evaluating the values of the coefficients {Aq, Ai, A2} of the distribution. 

Let us denote B C IR the range of the quantity X, i.e., the subset of the real line 
where the values of X have nonzero probability to occur. The functional form of the 
distribution is still given by the exponential function in Eq. ([2]), however with nonzero 
support only in IB, whereas the coefficients are to be determined by formulas like those 
in Eq. ([3]), again with R replaced by IB. It then follows, e.g., that for a variable which 
is known a priori to lie in a given interval, the maximum entropy distribution is not 
Gaussian, and the Gaussian approximation may be employed only if the standard 
deviation is small compared to range of the possible values of the quantity. 

Here we point out that having information about the range of variation may be 
expressed as a bias of the distribution toward a flat distribution in that range and 
the reasoning presented in [1] may be subsumed by the minimum Kullback entropy 
principle (mKE) [5l |6l [7]. The Kullback entropy, or relative entropy, or Kullback- 
Leibler divergence, of two distributions p{x) and q{x) reads: 

K[p\q\ = / dxp{x) log [p{x)/q{x)] . (5) 

According to the mKE, in order to find the distribution p{x) given a bias toward q{x), 
we should minimize the function: 



r r 

IC[p]=K[p\q] + Y,^k dxp{x)x''-Mk 



fc=0 



(6) 
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with respect to the function p{x), obtaining: 

p{x) =q{x)exp{-\Q- Xix - X2x'^}, (7) 

where the parameters Afc can be still (numerically) computed by using Eq. (jS]) . Eq. ([7]) 
represents the probability distribution satisfying the given constraints, but with a bias 
toward the distribution q{x), which, for instance, may contains the information about 
the range of the variable x. This information, which in the case of the MaxEnt is 
not explicitly taken into account, now it is naturally considered from the beginning. 
Remarkably, this is a different scenario from that covered in GUM-SI, i.e., when 
further information on the quantity is available, namely, the interval of values within 
which the quantity is known to lie is finite. 

Indeed, as mentioned above, if the standard uncertainty is sufficiently small with 
respect to the range of variation of the quantity, we can adopt a Gaussian probability 
distribution over the whole real axis and, thus, use the GUM-SI recommendation. In 
order to rigorously justify this statement, which has been qualitatively addressed in 
[3], we assess quantitatively how the knowledge of the range of variation infiuences 
the assignment of a probability distribution by considering the deviation of the mKE 
distribution from a Gaussian distribution, which would represents the MaxEnt solution 
in the absence of any information about the range of variation. The deviation from 
normality of the mKE distribution ([7]) may be quantified by its negentropy j8| : 

N[p] = ^[1 + log {2TTaj,)]~S[p], (8) 

where S[p] is the Shannon entropy ([T]) of the distribution ([7]). As for example, for a 
variable known to lie in a given interval [a, 6] C R, a < 6, that corresponds to a bias 
of p{x) toward the flat distribution: 

,(x) = ( (^-«)" ''"^t"''^ , (9) 

^ ' 1^ otherwise 

the negentropy ^ reads: 

N[p] = i [1 + log (27rf7|)] - log (6 - a) - Ao - Xix - X2{a% + x^) . (10) 

In the simplest case, namely when a; = and x G [—a, a], the dependence of the 
coefficients Ao and A2 is such that we have a scaling law for negentropy, which depends 
only on the ratio a/ ax ■ This is illustrated in Fig. [H where we report the negentropy 
as a function of aj ax for different values of ax ■ 

In conclusion, we have shown that the determination of the probability 
distribution of a variable for which we know the first two moments and its range of 
variation may be effectively pursued by using the mKE. Furthermore, the negentropy 
of the distribution may be used to quantify how much the mKE solution differs from 
the MaxEnt one, i.e. to assess how the knowledge of the range of variation influences 
the assignment of a probability distribution. Our analysis quantitatively supports the 
conclusions of Ref. ^ and rigorously justifies the use of GUM- SI recommendation also 
in the presence of further information on the range of variation of a quantity, namely, 
provided that its standard uncertainty is sufficiently small compared to the range. 
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Figure 1. Scaling of the negentropy of mKE distribution for zero mean value and 
variable known to lie in a symmetric interval [—a, a]. We report the negentropy of 
the distribution as a function of the ratio a/ ax for different values of the variance: 
: 0.5 (green squares), 1 (red circles), 2 (blue triangles). 
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